Apparatus and method for recognition of patient activities

ABSTRACT

A medication confirmation method and apparatus. The method of an embodiment of the invention includes the steps of capturing one or more video sequences of a user administering medication via a medication administration apparatus, storing the captured one or more video sequences, and analyzing at least one of the stored video sequences to confirm that the user has properly administered the medication, and has properly positioned the medication administration apparatus.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a non-provisional application that claims thebenefit of U.S. Provisional Patent Application Ser. No 61/498,576, filedJun. 19, 2011 to Hanina et al., titled APPARATUS AND METHOD FORRECOGNITION OF PATIENT ACTIVITIES, the entire contents thereof beingincorporated herein by reference. This application is also aContinuation in Part Application of U.S. patent application Ser. No.12/815,037, filed Jun. 14, 2010 to Hanina et al., titled APPARATUS ANDMETHOD FOR RECOGNITION OF PATIENT ACTIVITIES WHEN OBTAINING PROTOCOLADHERENCE DATA, which claims the benefit of U.S. Provisional PatentApplication Ser. No. 61/331,872 filed May 6, 2010, to Hanina et al.,titled APPARATUS AND METHOD FOR RECOGNITION OF PATIENT ACTIVITIES WHENOBTAINING PROTOCOL ADHERENCE DATA, the entire contents of each of theseapplications being incorporated herein by reference.

FIELD OF THE INVENTION

This invention relates generally to patient compliance in medicationadministration protocol scenarios, and more particularly to an apparatusand method for the collection, analysis and transmission of data relatedto patient movements related to such medication administration in orderto improve adherence with prescribed drug protocols in accordancetherewith. The invention further relates to a tracking and guidancetraining system for individuals taking prescribed medication, preferablyemploying computer vision and activity tracking, and may also provideincentives such as reminding a patient of the benefits of a particularmedication, providing the patient monetary points or other remunerationon the screen, and one or more reminders to take an appropriatemedication. The invention finally preferably relates to the use ofcomputer vision and activity recognition for determination of properadministration of injectable, inhaler-based, or other non-pillmedication administration in which proper position and action sequenceis important, but is equally applicable to pill based medicationadministration sequences.

BACKGROUND OF THE INVENTION

Dr Lars Osterberg, M.D. and Dr, Terence Blaschke have reported in theNew England Journal of Medicine, Adherence to Medication, (N Engl J Med2005; 353:487-97) 2005 an alarming lack of adherence to requiredmedication protocol, further noting that while the average rates ofadherence in clinical trials is categorized as “high”, this number stillcomprises only rates of 43 to 78 percent. Most importantly, the authorsnote “The ability of physicians to recognize nonadherence is poor, andinterventions to improve adherence have had mixed results.” Adherence,p. 487. The authors conclude “Poor adherence to medication regimens iscommon, contributing to substantial worsening of disease, death andincreased healthcare costs.” Adherence, p. 494. The Trend Repot Series,2008 Patient Adherence Update: New Approaches for Success, October 2008,report similar discouraging statistics. This broad range may possiblycontribute to the public confidence in the FDA approval process and theimportance of continued surveillance of a drug throughout the process.Furthermore, it may help to explain why, according to the Journal of theAmerican Medical Association (JAMA May 1, 2002), one out of every fivenew drugs that comes to market in the US is found to have serious orlife-threatening adverse effects—unknown or undisclosed at the time ofapproval. It is against this backdrop of poor adherence, and potentialdanger to patients, that the present invention operates.

It has been widely recognized that methods and systems for insuringproper medication ingestion or administration by individuals are veryimportant in defending against unnecessary sickness, deaths and otherproblems. Giving instructions and then letting patients fend forthemselves has been shown not to work particularly well. This is becauseit is not only the improper ingestion of medicines that is the primarycause of medical danger. Rather, an overall lack of sufficient patientguidance is also part of the problem. Further, the inability to confirma proper prescription regimen being provided to a user in the firstplace may cause a number of other problems with the use of suchmedication. As has been shown in regards to various public healthmedication administration situation, such as administration oftuberculosis medication by the WHO, Directly Observed Treatment (DOT)improves compliance of patients. Global Tuberculosis Control: A ShortUpdate to the 2009 Report, World Health Organization, 2009. As is shownin this report, funding for implementing DOT programs is high. Thus, theability to implement such a program with less of a financial burdenwould be desirable.

Traditionally, participants attend introductions and follow ups forclinical trials in-person. Other patients outside of the clinical trialsetting attempting to adhere to a particular medication protocolsimilarly are given a prescription and a particular set of instructionsfrom a prescribing medical provider or prescribing doctor, and thencompliance is measured at a next visit with that prescribingprofessional through traditional methods of counting unused medication,and patient interviews. Thus, data collection is similarly limited topatient visits, rather than on a daily basis. These old methods such aspatient questioning and medication counting have been proven to beinadequate measures of adherence and offer no information on dose timingand drug holidays (omission of medication for three or more sequentialdays).

Compliance technologies can increase the statistical power of clinicaltrials. Through the use of such technology, clinical events can beprecisely linked to medication use history. Captured data can be linkedto other sources such as EDC, patient diaries and data collected by thephysician. Technologies can create many possibilities for remote visitsand data capture. While smart packaging technologies exist such asRFID-enabled computer chip technology, smart blister packs and MEMS caps(microprocessor in a bottle cap), they are: a) invasive and need to bephysically attached to the medications; b) are non-conclusive regardingcompliance—a patient may activate the technology without ingestion ofthe medication; c) remain largely unadopted in clinical trials by thepharmaceutical and biotech companies due to their high cost; and d) takea longer time to implement. Further, electronic patient diaries allowfor ease of entry of data by a patient. These diaries, however, arestill subject to issues related to compliance with medication adherence.Thus, even if a patient is meticulous about entering information intothe diary, and thus complying with the requirements for data entry,there is still no guarantee that they are properly taking medication atprescribed times. Additionally, none of these techniques allow for themonitoring of non-pill related medications, such as the user of inhalersor injectable medication. These medication delivery systems requireadditional monitoring, as know when administration took place is notsufficient. Rather, positioning and other technique issues maysubstantially affect the proper delivery and efficacy of particularmedications.

Jo Carol et al. stated that “The most reliable method for researchpurposes, although not practical in a clinical setting, may be acombination approach that includes pill counts, patient self-report, andelectronic monitoring.” (Carol J. et al, Patterns to AntiretroviralMedication, The Value of Electronic Monitoring, AIDS, 17 (12), pp1,763-767, October 2003. To date, technologies alone have only been usedto monitor compliance rather than to encourage it. Furthermore, therehas been no comprehensive system provided that allows for the managementof multiple patients and multiple patient populations. While currenttechnology may allow poor compliers to be recognized, as will bedescribed below, the proposed apparatus and method of the presentinvention will help to encourage pharmaceutical compliance with non-pilldelivered medications, and tackle some of the problems that areencountered in the clinical trial process in particular, and themedication protocol monitoring problem in general.

A number of systems exist that provide instructions to a user regardingwhen to take a medication and records when the user indicates that amedication has been taken. U.S. Pat. No. 7,359,214 describes such asystem. A device is provided that provides instruction to a patientregarding medications to take. Furthermore, the system may provide amethod for determining that the prescription is appropriate given thepatient's conditions, and other medications he or she may already betaking. The system may monitor the dispensing of medicine in accordancewith a predetermined treatment protocol. While such a system providesmany improvements for easing a burden on the patient, this systemsuffers in many ways and in particular in ways relevant to theadministration of clinical trials and other active patient monitoring ofmedication adherence.

Most importantly, this system provides no mechanism for actuallyconfirming that a patient is in fact properly administering requiredmedication as required in a clinical drug trial, as prescribed by aprescribing physician in the case where adherence to a particularregimen may prove to be critical to efficacy of the prescriptionregimen, in various public health scenarios, in situations where failureto keep up a prescription regimen can potentially harm a population as awhole, such as the generation of antibiotic-resistant bacteria strains,in various disease management scenarios, or in home care situationswhere maintaining proper control of administering healthcareprofessionals is critical. Further, while the system may be sufficientfor one who is in full possession of their mental faculties, anyindividual who may have difficulty following directions, or one who isactively avoiding medication may still not be taking required medicationafter it is dispensed. Thus, participants may be forgetful, visuallyimpaired, or otherwise do not believe in the benefit of taking suchmedication, and may thus not properly log medication administration.Furthermore, as it applies only to pill based oral medication, thesystem requires preloading of various medications into a dispenser, andthus likely requires regular visits by an administering manager to besure appropriate medications are in fact properly loaded therein. It issurely possible that an inexperienced user may place incorrectmedications into the device, or may somehow provide incorrect dosagesinto the device. Additionally, for potentially more complex regimens,there is no method provided for insuring that a user is able to followsuch a protocol, and to thereafter confirm that the user has in facttaken all required medications in accordance with any providedinstructions or the like, or has taken the medications according to oneor more specifications or followed suggested procedures. Furthermore,this system is expensive and requires constant maintenance to confirmthat the various mechanical parts are in working order. Finally, asnoted above, the system has no application to non-pill basedmedications.

U.S. patent application Ser. No. 11/839,723, filed Aug. 16, 2007, titledMobile Wireless Medication Management System provides a medicationmanagement system employing mobile devices and an imaging technology sothat a user is able to show a pill to be taken to the system, and thesystem can then identify the medication. Patient histories are availableto an administrator, including various vital signs as measured by thesystem. Images may also be taken of the patient, provider, medicationcontainer or the like. While the system professes to ensure adherence toa protocol, the system only provides such help if requested by a user.There is in fact no particular manner in which to ensure actualadherence or ingestion of the medication, or the relationship ofadherence to the efficacy or safety of the drug over time. Whenrequiring adherence to a predetermined protocol for a clinical trial,this is particularly relevant. Similarly, there is no mention ofnon-pill based medications.

While adherence to medication in general is poor, requirements for useof inhalable medications, such as metered dose inhalers (MDI) and dryinhalers, have an increased burden in that simply confirming actuationof such an inhaler is insufficient. It is important to use inhalerscorrectly to get the full dosage and benefit from the medicine. By usingthe MDI correctly, medication has a better chance to reach the smallairways, increasing medication effectiveness. Using a good technique canalso help reduce the side effects of medications. However, use of propertechnique with such inhalers is difficult to instruct and monitor inthat between 28% and 68% of patients do not use metered-dose inhalers orpowder inhalers well enough to benefit from the prescribed medication,and 39%-67% of nurses, doctors, and respiratory therapists are unable toadequately describe or perform critical steps for using inhalers. Evenif patients are able to demonstrate correct technique duringconsultation with a health professional, they may not maintain thisstandard at other times. Improvement in patient compliance with therapywill require better doctor-patient communication, improved patienteducation, the tailoring of therapy to the individual and possible novelstrategies such as offering feedback to the patients on further theirlevel of compliance.

Additionally, existing systems fail to maintain an audit trail for postadministration review by a medical official or other clinical trialadministrator, and further cannot therefore confirm confirmation ofproper medication administration or population management.

Therefore, it would be desirable to provide an apparatus that overcomesthe drawbacks of the prior art.

SUMMARY OF THE INVENTION

In U.S. patent application Ser. No. 12/620,686, filed Nov. 18, 2009,titled Method and Apparatus for Verification of MedicationAdministration Adherence; currently pending, U.S. patent applicationSer. No. 12/646,383, filed Dec. 23, 2009, titled Method and Apparatusfor Verification of Clinical Trial Adherence, currently pending; U.S.patent application Ser. No. 12/646,603, filed Dec. 23, 2009, titledMethod and Apparatus for Management of Clinical Trials, currentlypending; and U.S. patent application Ser. No. 12/728,721, filed Mar. 22,2010, titled Apparatus and Method for Collection of Protocol AdherenceData, currently pending, the contents of these four applications beingincorporated herein by reference, the inventors of the present inventionhave proposed a system, method and apparatus that allow for completecontrol and verification of adherence to a prescribed medicationprotocol or machine or apparatus use in a clinical trial setting,whether in a health care provider's care, or when self administered in ahomecare situation by a patient.

These applications present the only medication management system thatmay determine whether a user is actually following a protocol, provideadditional assistance to a user, starting with instructions, videoinstructions, and the like, and moving up to contact from a medicationadministrator if it is determined that the user would need suchassistance in any medical adherence situation, including clinical trialsettings, home care settings, healthcare administration locations, suchas nursing homes, clinics, hospitals and the like, and in clinical trialsettings.

The inventive solution, in accordance with one or more embodiments ofthe invention, may provide a webcam software solution, for distributionby medical professionals to provide a training system for trainingpatients to properly administer their inhalable, injectable, or othernon-pill based medication, to automate direct observation of medicationadministration of inhalable, injectable or other medications, and toprovide an audit trail of medication adherence and patient behavior. Theinventive system may visually and audibly track medication adherence toinhalable, injectable and other medication during training and actualmedication administration in clinical trials or other medicationadministration scenarios on webcam-enabled laptops, tablets, smartphonesand other platforms without real time human supervision. The inventivesystem may visually and audibly recognize a fixed series of actions,each comprising part of the medication administration process.

In accordance with an embodiment of the present invention, a motioncapture procedure for capturing motion information related to theadministration of injectable, inhaler-based, or other non-pill basedmedication, may be utilized in accordance with one or more of theinventions noted in the above-referenced applications. Therefore, inaccordance with an embodiment of the present invention, a method andapparatus may be provided for analyzing captured patient motion data,transmitting such captured patient motion data to a remote location (orprocessing such captured information locally, in whole or in part),receiving information from a remote location (or from a processormaintained locally) and providing information to the patient aspreferred in accordance with the present invention.

Further in accordance with an embodiment of the present invention, oneor more predetermined motion sequences may be determined and correlatedto one or more corresponding medication administration instructions foradministration of injectable, inhaler-based, or other non-pill basedmedication. These predetermined motion sequences may be provided as anumber of generic motion sequences, as one or more customized motionsequences, or a combination of both. The group of predetermined motionsequences may comprise a motion language that may be applied to one ormore different medication administration sequences, including injectablemedication administration, inhaler-based medication administration,other non-pill based medication administration and the like, and otherhealthcare related processes, such as hand washing or the like,medication administration personnel acting as prescribed or the like,thus allowing for an easy to program generic medication administrationsequence, but also allow for customization where appropriate andnecessary. These programmed motion sequences may then be applied inaccordance with the inventions noted in the applications above.

In yet another embodiment of the invention, one or more methods may beprovided for confirming that a user is properly performing one or moreof these predetermined motion sequences. Thus, as a user positionsthemselves or an object before an image capture device, a display mayindicate to the user whether the position, distance, and/or skew andangle are correct. If not, the user is preferably provided withindications as to how to correct any one or more of these issues. Themotion sequences may include capture of use of an inhaler, injectabledevice or the like.

In a still further embodiment, one or more audio cues may also beemployed. Thus, for example, in the case of an inhaler-based medication,audio monitoring of sound from both the inhaler and patient may beperformed and used to further confirm that the patient has in factproperly administered the medication. Therefore, not only maypositioning of the inhaler in the correct location and relative angle beconfirmed, but activation of one or more inhaler mechanisms, and theinhaling of the medication by the patient upon such activation may alsobe confirmed. Similar monitoring may also be performed with injectableand other non-pill based medications.

The system in accordance with one or more of the various embodiments ofthe present invention may also be applicable to training situationswhere the user is provided with various feedback instructions related totraining to properly administer medication in a clinical trial or otherdisease management scenario. In accordance with various embodiments ofthe present invention, for example, when applied to an inhaler, apatient may be requested to shake the inhaler before use. When appliedto an injectable medication, the patient may be requested to confirmrefrigeration or confirm proper sanitization of an injectable tip withan alcohol swab or the like, confirm that the needle is not bent, orthat an injectable solution or medication has not changed color, orotherwise become spoiled in a manner that is visually detectable. Whileonline training and instructions may be available currently, theinteractive nature providing feedback to the user regarding their useand following of protocol is critical in improving adherence and patientaction.

Still other objects and advantages of the invention will in part beobvious and will in part be apparent from the specification anddrawings.

The invention accordingly comprises the several steps and the relationof one or more of such steps with respect to each of the others, and theapparatus embodying features of construction, combinations of elementsand arrangement of parts that are adapted to affect such steps, all asexemplified in the following detailed disclosure, and the scope of theinvention will be indicated in the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

For a more complete understanding of the invention, reference is made tothe following description and accompanying drawings, in which:

FIG. 1 is a flow chart diagram depicting a method in accordance with anembodiment of the invention;

FIG. 2 is a flowchart diagram depicting a video sequence generationmethod in accordance with an embodiment of the invention;

FIG. 3 is a flowchart diagram depicting a video sequence capture methodin accordance with an embodiment of the invention;

FIG. 4 is a depiction of a positioning process in accordance with anembodiment;

FIG. 5 is a depiction of another positioning process in accordance withan embodiment of the invention;

FIG. 6 is a depiction of yet another positioning process in accordancewith an embodiment of the invention; and

FIG. 7 is a depiction of a motion tracking process in accordance with anembodiment of the invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

The invention will now be described making reference to the followingdrawings in which like reference numbers denote like structure or steps.Referring to FIG. 1, a data flow overview in accordance with theoperation of an embodiment of the present invention is shown. Inaccordance with this embodiment of the invention, information about aparticular drug to be the subject of a clinical trial, to be employed ina public health or disease management situation, or the like, or othermedication administration program or prescription may be provided in adatabase 105, and existing industry medication information databases 110are preferably employed to access prescription, interaction,application, and other available information about any number ofproposed prescription and non-prescription medications and theirpossible interaction with the clinical trial or other medications.Further, patient medical records 115 may be used, and as will bedescribed below, are used in conjunction with the industry medicalinformation and a medical professional's prescribing expertise toconfirm that a patient is a good candidate for such a clinical trial, ormedication administration program. These databases may be accessed in amanner known to one of ordinary skill in the art.

Once confirmed, a medication administration regimen in accordance withthe clinical trial or other prescription requirements such as in apublic health, medical practice environment or the like may beprescribed and entered into the system of the invention at 120. Onceentered into the system, a particular prescription regimen may cause aset of user instructions, various training sequences and the like 125 tobe generated and transmitted to an apparatus provided to a patient inaccordance with an embodiment of the invention for access to the systemof the invention. Such an apparatus may comprise a custom designed videoand audio capture, analysis and transmission apparatus, a smart phone orother mobile device including a camera or other video and audio captureapparatuses, a netbook, laptop computer, desktop computer, tablet deviceor the like, or other computing appliance allowing for the display ofinstructions to a patient, and allowing for the eventual capture,analysis and transmission of video, audio and other analysisinformation. When installing software on a user's own hardware system,it is preferred that the software detect and otherwise test or determinethat the hardware attempting to be utilized by the patient is sufficientto implement the invention and is sufficient to run a software packageprovided in accordance with the invention. Thus, the software may checkthat a camera includes sufficient resolution, that a memory of thedevice is of sufficient size to allow for sufficient captured videostorage, that audio may be properly captured, and that the transmissionsystem includes sufficient bandwidth to transmit and receive capturedvideo, audio, video instructions and the like.

In a clinical trial or other administration settings, patientinstructions and various training sequences may be varied for differentusers to determine the best set of instructions, or may be varied basedupon demographics, experience, or other factors that may requiredifferent types of instructions to be provided. It is furthercontemplated in accordance with an embodiment of the invention thatmultiple clinical trials or patient populations may be managed by amanager in accordance with the invention so that the inventioncontemplates a medication administration system that allows for a singlepoint of management for all clinical trials or patient management groupsassociated with a particular manager or the like. Such managementtechniques in accordance with the embodiment of the invention mayfurther be applied to various public health situations, diseasemanagement scenarios and the like.

Such user instructions and training sequences may include generalinstructions about the particular medication subject to the currenttrial or medication administration protocol, methods for administration,warnings about side effects, and concerns about drug interactions withcommon substances or medications, or other medications prescribed to thepatient by the system or by another medical service provider. It iscontemplated in accordance with an embodiment of the invention that suchset of user instructions may be interactive, allowing a user to viewadditional information about such instructions or prescriptions asdesired. These instructions may comprise written, audio or videoinstructions provided to the user on a display of the user apparatus. Itis further contemplated that such instructions may indicate one or moremovement sequences to be associated with a corresponding one or moremedication administration sequences. A more in-depth description of theinformation provided at step 125 is depicted in FIG. 2.

As is shown in FIG. 2, the generation and provision of user instructionsas set forth in step 125 first comprises the step of receiving asequence of required instruction steps at step 205. This sequence may bedetermined as described above in step 120. The system then may confirmwhether one or more of the instructions steps require the conveyance ofinformation to a patient at step 210. These conveyance steps maycomprise a more conventional instruction step, such as the display ofwritten information, comprise a more advanced instruction step, such asthe conveyance of audible information, video instructions or the like,or may comprise an interactive instruction step, such as an interactiveinstruction sequence displaying a desired sequence of information to apatient, and then monitoring and confirming whether the patent hasproperly administered the medication. Various feedback mechanisms may beprovided to allow the patient to try multiple times to perform properadministration, and may also provide varying encouragement orinstructions to confirm that administration training has been performedproperly. Thus, such an instruction and training sequence may includethe eventual capture of video, audio and other information from theuser. Therefore, at step 215, it may be determined whether one or moreof the instruction steps will require the capture of information fromthe user, thus comprising an advanced interactive training sequence.Thereafter, each of the training steps requiring capture of videoinformation from a user is confirmed at step 220. If no further videocapture is required, and therefore various training or other interactivesequences have been completed, processing for step 125 then ends at step250. If it is determined that the capture of video and/or audioinformation will be required at step 220 for the current training step,then processing passes to step 230, and various instructional video,audio and other sequences may be provided to the user in aninstructional sequence format.

After being shown a particular instructional sequence, preferablyapplicable to a particular step of a medication administration protocolsequence, then processing passes to step 235 where the user may beprompted to perform a particular action or sequence of movements. Theuser may request to be re-shown these sequences as many times asnecessary, and may also include audio or other instructions, so that theuser is provided with a training sequence, thereby reducing variabilityof future performance of that action. When preparing to perform theseactions, an alert system may be employed to warn the patient of anyissues that may interfere with the proper capture of video and/or audioinformation, as may take place similarly when actually administering themedication. Thus, the user may be encouraged to properly perform thesesequences, thus acting as an interactive training module.

Thus, the user may be notified if they are sitting in a manner in whichtheir actions cannot be properly captured, if they are blocked from thecamera, the light conditions are insufficient, if an object they areholding is in an improper location, or the like. As is shown in FIG. 4,a box 410 may be provided on a display viewable by a patient using thesystem. A representation of the patient's face may be shown in aposition relative to an optimal filming position for the use of, forexample, an inhaler for medication administration. Thus, while facialrepresentation 400 a is properly positioned, facial representation 400 bis positioned to the left of the box, while facial representation 400 cis positioned down and to the right of the box. A similar positioningsystem may be provided for an injectable medication, the position of apatient body part being provided in place of the facial positioningdescribed above. Thus, not only may proper positioning be determined,but use of the proper body part may also be confirmed. In practice, thebox may be made a red or other warning color until proper alignment isachieved (including if a user or desired user body part is notpositioned fully within a screen, the user is too close or far from thecamera, or for any other reason), at which time the box may change togreen or other appropriate color. Further, audio clues may also be givento the patient, such as increasing frequency beeping as the optimalposition is approached. Thus, in accordance with an embodiment of thepresent invention to be employed for inhaler medications, the user isprovided with immediate feedback on their position and the ability oftheir actions to be properly recorded and analyzed. As the userinteracts with the system of this embodiment of the invention, such ascheme may be employed to provide continuous feedback to the user, andthus indicating whether the system is able to properly capture and/pranalyze the actions of the user. If time is passed and the user isunable to properly position themselves, or to properly perform desiredactions, additional guidance may be provided to the user in order toremedy such a situation, including but not limited to directionalindications, voice commands, video images of proper technique, etc.

In addition to properly positioning the patient, proper positioning ofone or more objects, either absolutely or relative to another body part,may be determined, such as positioning an inhaler relative to the mouthand face of the user, an injectable medication delivery device relativeto the body part of the user to receive the injection, or the like forimaging and processing in accordance with an embodiment of theinvention. As is shown in FIG. 5, an inhaler 500 may be indicated asproperly positioned by a box 522, the box being green, for example, asin the description of FIG. 4. Such an object, however, is more likely tobe improperly positioned not only left to right and up to down, but alsoin distance to the imaging apparatus, in accordance with one or morelimitations of the imaging device, such as the resolution thereof, lowlight positions, and the like, and any affect such resolution might haveon the ability of the imaging device to identify shape, color text orother coding, or the like associated with the object being imaged. Thus,if positioned too far away from the imaging apparatus, a sequence ofboxes 510, 511, 512 and a small representation of inhaler 500 may beprovided to alert the user to move the inhaler closer. If the inhaler isnot only too far away, but off center, boxes 520, 521, 522 may beprovided to guide the user to move the inhaler into proper positionabsolutely and relative to the mouth and face of the user. Similarfunctionality may be provided for positioning an injectable apparatusrelative to a user body part to receive the injection, includingrelative angle and distance to the body part. By properly positioningsuch a device, the system may be employed to confirm the identity ofsuch a medication, employing shape, color, labeling, and the like. Inaddition to determining identity of the medication, such processing maybe used to determine safety of the apparatus, such as whether an inhaleror injectable device may have been damaged or tampered with. Further,the medication may be observed to determine any change in color or othercharacteristic of the medication that may suggest spoilage, impropermedication, counterfeit medication or the like. The apparatus, inaccordance with an embodiment of the invention, may thus ask the user tomove the inhaler or injectable device closer to or further away from theimaging apparatus, may change an ambient light sensitivity of theapparatus, or may otherwise change details of the image capture. Asnoted above, both color and audio prompting may be provided.

To the extent that positioning and orientation of the inhaler,injectable medication administrator or the like when being used isimportant, a similar system may be employed. As is shown in FIG. 6, aset of concentric circles 610 a-e may be provided to aid in thepositioning of an inhaler 600. A center circle 610 e may be providedwith a solid center (not shown) upon proper placement of the inhaler.These circles may move as the boxes in FIG. 5, and may further use colorand/or audio prompts to instruct the user. Further, as images of inhalerpositions and orientations, or inhaler and hand positions andorientations, are to be captured and analyzed, the system may alsopreferably indicate not only proper positioning, but actual acquisitionof a correct position and orientation sequence. In accordance with anadditional embodiment of the invention, such recognizable positioningand orientation may further comprise a sequence of gestures andapparatus movement and orientation employed to ensure that the patientproperly administers their medication. In accordance with anadministration process, as noted above, the patient may first be trainedto show a particular medication administration device or apparatus intheir hand to the camera for imaging and recognition. The patient maythen be asked to place the apparatus at an appropriate administrationlocation, such as against the mouth in the case of an inhaler apparatus,or at a particular body part location in the case of an injectablemedication. Thereafter, actuation of the apparatus, through the processof monitoring movement and audible cues may be employed. Thus, through apredetermined sequence of actions that are captured, imaged andanalyzed, evidence of proper administration can be recorded andanalyzed.

Furthermore, in accordance with one or more embodiments of theinvention, various additional aspects of medication and/oradministration may be checked and confirmed. Thus, the system may employsuch computer vision and activity recognition to determine a liquidcolor, liquid consistency or clarity, potential existence of particles,perhaps suggesting a spoiled medication, bubbles in the liquid,suggesting improper handling, in an injectable administration system.Through the use of the system, a number of administrations can betracked, and a liquid or other level may be used to confirm the count,thus potentially allowing for the addition ordering of furthermedication, or other counting of inhaler administrations without theneed for expensive inhaler units. Also, dosage settings, if applicableon an injectable pen or other apparatus may also be confirmed beforeadministration.

Furthermore, as is shown in FIG. 7, when tracking the movement of amedication administration apparatus 700, it is preferable to depict to apatient whether they are holding the apparatus at a correct orientation,when the apparatus is in transit, or positioned at the administrationsight. Thus, as is show in FIG. 7, an administration apparatus 700 isindicated to be reoriented from a horizontal to a vertical orientationthrough movement in the direction noted by arrows A. A set of guidancetracks 710 a, 710 b may be displayed to a patient and successiveapparatus positions and orientations may be superimposed thereon. As theapparatus moves along the proscribed path, concentric circles such asthose depicted in FIG. 6 may be employed to confirm proper location andorientation. Thus, in accordance with an embodiment of the invention, avirtual path may be shown to the user to ensure that the proper methodof medication administration is followed. As noted above, color and/oraudio sequences may also be employed. Similar positioning informationmay be processed relative to an injectable medication.

Therefore, in accordance with one or more of the positioning assistanceschemes noted in FIGS. 4-7, a patient may be guided to properly presentthemselves or an object to an image capture device for capture andinterpretation during the noted training phase, or (as will be describedbelow) during a particular medication administration phase. Any of thedisplay and notification techniques noted in any of these Figures may beused in any of the other Figures, in accordance with various embodimentsof the invention. Further, these positioning techniques may be employednot only during initial training, but during any subsequent systemprocess employing video image capture of people, objects, or any otherentity to be imaged, or the use of audio information.

Referring back to FIG. 2, at step 240 these motions of the user may becaptured and confirmed as being correct by one or more appropriatecomputer vision techniques, individual review by a human, or otherappropriate determination process. If not correct, processing may returnto step 230 to provide the instructions and example sequences again tothe user. Therefore, in accordance with the invention, repeatedinstruction may be provided to the patient until training can beconfirmed that the patient has performed the desired sequence correctly,thereby aiding in limiting future variability in the actions taken bythe patient during administration. Such instruction may take the form ofanalysis of a recorded user action, and comments on what the user may bedoing wrong, and how this action may be improved. Once the user hasreceived sufficient instruction, and it is therefore determined that theuser has performed the action in a manner that is sufficiently similarto the instruction set, and substantially consistent over a number ofperformances of the action, processing then passes to step 245 where itis determined whether there are additional training steps to bepresented, and therefore additional video sequences to be captured. Ifso, processing returns to step 220 for further processing. If not,processing ends at step 250.

Referring back to the lower portion of FIG. 1, the horizontal lineindicates a time for patient administration of medication. At such time,the user may be notified to take their medication through any desirablecommunication and notification system, including text messaging, email,telephone call, automated calendar reminder or the like. While notexplicitly shown, first, preferably the identity of a user is confirmedthrough the use of a facial recognition sequence, other biometricidentification sequence, or other password identification system. Uponrecognition of the individual, the system may display one or more dataregarding the individual, such as, by way of example only, name, patientstatus, medication to be administered, calendar indicating to thepatient when medication has been administered and if any administrationtimes have been missed, and, selectively, a score indicative of a levelof compliance of the individual with the medication protocol, ifdesired. Once identified and notified of a type of medication to beadministered, the patient may display a medication administrationapparatus, such as an inhaler, injectable apparatus, or other medicationform (including a pill bottle, pill, or the like) to confirm that themedication is correct and is the currently prescribed medication to betaken through the use of text recognition, medication recognition,barcode or other code reading of one or more unique identifiers from theadministration apparatus, pill bottle or the like, or other appropriatemedication recognition scheme. The user may alternatively be shown avirtual medicine cabinet with visual or textual indications of one ormore medications to be taken at a particular time. Imaging of one ormore of such medication apparatuses may then match a medicationapparatus provided by the patient to one or more of the pills in thevirtual pill box. Thus, the patient is not only allowed to have aparticular medication apparatus imaged, but also may be given a visualrepresentation of medications to be taken, medications that have alreadybeen taken, and a visual picture of one or more additional medicationsto look for if the patient is confused or is not immediately able tolocate all of the required medication. Such a display may further act asan additional incentive program for the patient to properly take theirmedication, and may in turn give a patient other incentives, such as arunning score, payment information or other point systems if the patientis to be rewarded for properly taking medication. Thus, credit to buyinformation from a website or store may be provided. For children,various animations may be provided, and pocket money or other creditsmay be provided to purchase items online or through one or more storesfrom supporting merchants may be provided. The display of suchinformation may assist in convincing the patient to continue to properlytake medication. This sequence of steps therefore acts as an audit traileach time a medication is taken, that can be reviewed later, to ensurethat a patient is properly following a regimen. Any of the positioningschemes depicted in FIGS. 4-7 may be employed.

Additionally, after confirmation or failure of confirmation of suchadministration, the user may be provided with a progress reportregarding how they have performed over time, and further providingencouragement for future adherence. Additionally, notice of a nextadministration time may be provided, along with one or more messagesfrom a healthcare provider regarding protocol changes, or other desiredinformation.

Furthermore, use of a combination of visual and/or audio cues may beemployed to further determine sequence and timing. Thus, not only shouldan inhaler be properly positioned, for example, but during use, aninhalation by the patient should occur immediately after actuation ofthe inhaler. Thus, by visually and/or audibly confirming firstactuation, and then inhalation, this sequence of actions can beconfirmed. Sound and visual signatures related to each of these actionsmay be employed to improve a confidence with which the system is able toconfirm proper administration. Similarly, an injectable may need to beproperly positioned and maintained in a particular position afteradministration, such as maintenance of a needle after actuation of theinjection mechanism for a predetermined period of time.

In accordance with the invention, confirmation of patient adherence tothe prescribed administration schedule for the medication as prescribedby the clinical trial or other prescription regimen may be determined.While such confirmation may take a number of forms, in accordance withthe invention, a preferred method for such confirmation may includecapturing a video and audio sequence of the patient actuallyadministering the medication. In a further preferred method, such asequence for such confirmation may include employing a facialrecognition sequence or other biometric confirmation that a particularpatient is in fact receiving treatment, but may also provide for theability to obscure the face or other identifying feature of a user, orotherwise encrypt such information to allow for the storage and use ofsuch images while protecting the identity of the patient, a techniquethat may be beneficial when a medication administration manager isproviding a general report about a clinical trial, and not trying toremedy a situation with a particular patient, or in particular in apublic health or disease management scenario. Activity recognition,gesture recognition or other feature for determining whether aparticular subject movement meets a predefined movement sequence may beemployed to be sure that the patient is properly taking prescribedmedication.

Referring next to FIG. 3, a method in accordance with an additionalembodiment of the present invention for performing audio and videocapture and recognition of adherence to a prescribed protocol isdescribed, as set forth in steps 130 and 135 of FIG. 1. In FIG. 3, apatient may first log into the system of the invention at step 305,employing the facial recognition, biometric recognition, password entry,or other patient identification method, and at step 310 propermedication is confirmed as noted above, through the user of bar codereading, text recognition, visual recognition employing video or stillimage recognition, or other medication recognition technique. Thepatient may be reminded to log onto the system to take their medicationthrough any type of reminder, such as a text message, email, phone call,automated alarm or the like. Of course, any of the positioningtechniques previously described in reference to FIGS. 4-7 may beemployed. Next, at step 315 it may be confirmed that the processinvolved will include one or more information capture steps, and at step320 it may be determined whether these information steps will includevideo capture. If not, video processing ends after storage of anynon-video information. (Alternatively, steps 315 and 320 may be excludedif it is determined that each confirmation sequence may employ videocapture, then video processing may pass directly to step 325, asdescribed below.) If it is confirmed at step 320 that one or more stepswill include video and/or audio capture, processing then passes to step325 where the user may be prompted to perform one or more predeterminedactions, these actions being captured. Positioning of the inhaler,injectable medication apparatus, or other medication may be performed inaccordance with any of the techniques as described previously inreference to FIGS. 4-7. Such recognition in the case of an injectableadministration apparatus may also comprise confirming relationship ofthe injectable administration apparatus and a prescribed body part,proper actuation of the administration apparatus, maintaining theadministration apparatus in the location for a predetermined period oftime, and perhaps proper post administration action, such as cleaningand storing the apparatus, refrigerating the apparatus, cleaning aninjection site and the like. Further, voice recognition may be utilizedto allow the user to enter commands, and an audio output may be providedfor aiding the user in properly adhering to instructions from thesystem. Additional audio cues may be recognized, such as upon visualconfirmation of administration of an injectable or inhalable medication,audio signatures may be employed in order to determine whetherinsufficient pressure may have been used, or whether a sufficient orextensive period of time has passed from actuation to inhalation. Propercapture of patient actions is very important as the patient onlyadministers the medication once per capture period.

Video capture analysis may then begin at step 330, such analysiscomprising analysis of the newly captured video and/or audio, asprovided as noted above with respect to FIG. 2. At step 335 it may bedetermined whether the action has been properly captured, and whetherthe captured action has been properly analyzed by the system. Variousincentives may be provided to the patient to encourage them to taketheir medication properly. Thus, in addition to providing variousreminders to a patient as is known in the art, points, monetary or otherincentive may be provided to the user for actually having medicationadministration confirmed. Further proper administration with lesserrors, etc. may be rewarded more highly, thus giving incentive for thepatient to concentrate on administration issues and to attempt to havesuch administration be as accurate and consistent as possible. Suchincentives and medication tracking may be used to determine futurecourses of treatment or payment. For example, if a patient consistentlyfails to take medication as required, perhaps a different course oftreatment requiring fewer medication administrations may be better forthis patient. Alternatively, if a medication requires a consistentadministration and is very expensive, failure to comply withadministration instructions may be cause for an insurance company,prescribing doctor or the like to not renew such a prescription for thepatient, thus saving money in a situation where the money was beingwasted because of lack of compliance.

If it is determined that administration of the medication did not takeplace properly, processing may return to step 325 and the user may beonce again prompted to perform the action. Of course, if this processinvolves actual administration of inhaler or injectable medication, itmay not be proper to request re-performance of the action, unless it canbe determined that the user did not actually administer the medication.If the action has been properly captured, and is able to be analyzed,processing passes to step 345 where it may be determined whetheradditional captures are required. If so, processing returns to step 320.If no further captures are required, processing ends at step 350 wherethe various captured video sequences are stored. These stored sequencesmay also be made available for human review and involvement, when it isdetermined that this would be beneficial.

Therefore, in accordance with various embodiments of the invention,because a video image of the patient actually administering an inhalableor other medication (or other method of medication administration,including but not limited to injections, dialysis, and any othermedication administration procedure) may be captured and analyzed,actual confirmation may be achieved, rather than simply relying on thepatient to state that a particular medication was administered. Such avideo image may be captured or stored in any appropriate format given aselected type of activity or gesture recognition that is employed inaccordance with a particular embodiment of the invention. Such mayinclude full video, biometric data points, recording of movement of anarticle, such as a bracelet or the like, affixed to the patient oradministrator, use of mapping to provide a stick figure or other bodymovement tracking technique, or gesture or activity recognition todetermine movement or the like. The user may be encouraged to use aparticular sequence of movement to be confirmed that they are properlyadministering the medication according to the protocol, thus reducingthe possibility of the potential appropriate movements considered to be“correct.” Or, as noted above, capture of customized video sequences maybe performed so that the user is more likely to repeat these sameactions. Indeed, various instructional videos or other appropriatetraining may be provided to a user to insure they properly administerthe medication.

This captured adherence information may be provided to a healthcareprovider, clinical trial manager or the like through a dashboardallowing for the review of information about an individual patient,entire population of patients, or demographically relevant information.Such information may be provided to easily notify the healthcareprovider, clinical trial manager or the like of problem patients,demographic groups, medications or the like. One or more dashboards orother reporting mechanisms may be employed as described in copendingU.S. patent application Ser. No. 13/189,518, filed Jul. 24, 2011 toHanina et al., titled “Method and Apparatus for Monitoring MedicationAdherence”, the entire contents thereof being incorporated herein byreference. Thus, any adherence or other information obtained inaccordance with the present invention may be provided to one or moreindividuals in accordance with one or more methods or systems asdescribed in the '518 application.

Through the use of training as described above, a type of administrationlanguage may be generated, allowing for extension to other patients, andalso allowing for interpretation of reason for differences from apredefined sequence by a patient. Thus, if a patient performs an actiondifferently over time, this difference may provide insight to a reactionto a medication, changes in the patient's medical condition, or thelike. It is further anticipated that analysis of large numbers ofpatients will allow for a more flexible system that may recognize moreof a patient's movements, and thus may improve the ability of the systemto function properly.

Therefore, in accordance with an embodiment of the invention, a user mayperform a predetermined sequence of actions designed to ensureperformance of medication administration. Thus, by way of example only,for an inhaled medication as noted above, the user may be asked to firstshow a medication and may then be prompted to position the medicationadministration apparatus relative to their mouth in a desired manner.Next the user may be prompted to administer the medication, the actionof administration being captured on video and audio, and beinginterpreted to confirm that the medication has been properlyadministered. Of course, in accordance with this embodiment of theinvention, other action sequences may be employed, and may be mixed withother actions to be performed by a patient or caregiver. Thus, butdefining a medication adherence protocol as a single or sequence ofgestures that may be recognized by a processing system, the accuracy ofconfirming that a patient has actually taken a particular medication isimproved. Through an interactive learning process, the processing systemmay also learn patient behaviors to be more accurately determinemedication adherence, and to remove some of the potential falsepositives or false negatives. If a caregiver is involved, it iscontemplated that the caregiver be provided with a number of gesturesindicative of particular actions to be taken, and use of these gesturesprompting the system to confirm that these actions are in fact beingtaken. Thus, a full audit trail of not only the patient, but also thecaregiver may be determine, such as whether they approached the patientat the correct times, or that they washed their hands when approaching.

Further uses of the video capture sequences may also be employed,including video capture of responses to questionnaires about currentpatient states of discomfort, informed consent, example of questions tobe asked, video transmission of such questions and the like. The patientmay be able to send a video message, pointing to a particular pain orthe like, and may include an audio portion as well. Time stamp markersmay also be captured to confirm that the user is taking their medicationat appropriate times and a number of times a user has taken a particularmedication, to confirm whether there are substantial delays betweeninstruction and administration, or for any other time sequencedetermination. Furthermore, other behavioral markers, such as, by way ofexample only, shaking hands indicating a particular ailment, or othermovements by a patient that may give a hint as to the physical or mentalstatus thereof Additionally, if the user is taking medication that isimproper, or they have already taken, a warning may be provided to warnthe user to stop medication administration immediately.

In accordance with various embodiments of the invention, whenconsidering administration of an inhalable or injectable medication,analysis of adherence video sequences may be employed to determine alikelihood that a patient has actually administered their medication.Thus, based upon video and audio cues determined related to positioningand use of the medication administration apparatus, it may be determinedthat the patient is having problems properly positioning the apparatus,and therefore the system is unsure that the patient has administered themedication properly. Low confidence in proper administration based uponfailure to properly position the apparatus, failure to record audiosignals indicative of proper administration or the like may be employedto determine whether a patient should be retrained, via the automatedtraining system described herein, by automated contact, or by individualpersonal contact. This determination of low confidence ofadministration, even if it is ultimately determined that administrationlikely took place, may still be utilized to determine whether trainingor other actions may be taken. Such confidence levels may be used, inaccordance with a desired algorithm or the like, to provide an overallpicture of medication administration by a patients or group of patients,thus allowing for intervention, encouragement, training or the like tobe provided when it appears that actions are changing, but notnecessarily waiting until a critical issue is discovered.

It is further contemplated that the method and apparatus of theinvention allow for integration with one or more audio or videoconferencing systems, thus receiving and/or providing information therethrough. Thus, a user may employ a standard video conferencing tool orsystem, and have this information be coupled to a mobile or other devicebeing used in accordance with an embodiment of the present invention.

Therefore, in accordance with the invention, a method and apparatus areprovided that allow for the automated confirmation of adherence toadministration protocol for medication, and provide for a mostsophisticated method for confirming and studying methods ofadministration of such prescription medication.

It will thus be seen that the objects set forth above, among those madeapparent from the preceding description, are efficiently attained and,because certain changes may be made in carrying out the above method andin the construction(s) set forth without departing from the spirit andscope of the invention, it is intended that all matter contained in theabove description and shown in the accompanying drawings shall beinterpreted as illustrative and not in a limiting sense.

It is also to be understood that this description is intended to coverall of the generic and specific features of the invention hereindescribed and all statements of the scope of the invention which, as amatter of language, might be said to fall there between.

What is claimed:
 1. A medication administration confirmation apparatus,comprising: a display associated with a local device for displayingaudio-video instructions for instructing a user to properly administer amedication in accordance with a sequence of interactive instructionsteps indicative of proper medication administration employing amedication administration apparatus, at least one of the audio-videoinstructions instructing the user to follow one or more of the sequenceof interactive instruction steps to properly position the medicationadministration apparatus relative to a user of the medicationadministration apparatus; an audio-video capture device associated withthe local device for capturing one or more audio-video sequences of theuser administering medication following a display of one or moreaudio-video instructions encouraging adherence to the sequence ofinteractive instruction steps encouraging properly positioning of themedication administration apparatus for proper medicationadministration; a memory for storing on the local device the capturedone or more audio-video sequences; and a processor operable to performoperations on the local device for providing a user real time feedbackduring medication administration on how to properly administermedication and position the object associated with proper medicationadministration as the user positions the object before the video capturedevice, wherein the operations comprise: outputting to the display oneor more audio-video instructions for instructing a user to properlyadminister medication in accordance with the sequence of interactiveinstruction steps indicative of proper medication administrationemploying the medication administration apparatus, at least one of theaudio-video instructions instructing the user to properly position themedication administration apparatus associated with medicationadministration, receiving, from the audio-video capture device, one ormore audio-video sequences of the user administering medicationfollowing the displayed one or more audio-video instructions encouragingproper medication administration in accordance with the sequence ofinteractive instruction steps, including at least properly positioningof the medication administration apparatus associated with medicationadministration, and in response to receiving the one or more audio-videosequences, automatically analyzing in real time motion of the userand/or the medication administration apparatus within at least one ofthe received audio-video sequences to confirm whether the user hasproperly administered the medication in accordance with the sequence ofinteractive instruction steps using the medication administrationapparatus; determining, from the analysis, that the user has notfollowed one or more of the steps of the sequence of interactiveinstruction steps, and that medication has not been properlyadministered, and responsive to determining that the medication has notbeen properly administered, outputting to the display while the user isadministering the medication an additional one or more of theaudio-video instructions to the user to encourage following of the oneor more steps of the sequence of interactive instruction steps toconfirm proper administration of the medication.
 2. The medicationconfirmation apparatus of claim 1, wherein the processor is furtheroperable to output to the display a prompt to the user to perform aparticular sequence of actions in response to the displayed audio-videoinstructions to be stored as the one or more audio-video sequences. 3.The medication confirmation apparatus of claim 2, wherein the particularsequence of actions is designed to assist in determining propermedication administration.
 4. The medication confirmation apparatus ofclaim 1, further comprising a data transmission system operable totransmit one or more of the at least one stored audio-video sequences toa remote location for further analysis.
 5. The medication confirmationapparatus of claim 4, further comprising a data reception systemoperable to receive further information indicative of whether the userhas properly administered the medication following the displayedaudio-video instructions from the remote location.
 6. The medicationconfirmation apparatus of claim 1, wherein analyzing at least one of thereceived audio-video sequences comprises confirming proper positioningof an inhalable medication administration apparatus following thedisplayed audio-video instructions on the display of the local device.7. The medication confirmation apparatus of claim 1, wherein analyzingat least one of the received audio-video sequences comprises confirmingproper positioning of an injectable medication administration apparatusin real time following the displayed audio-video instructions.
 8. Themedication confirmation apparatus of claim 1, wherein the processor isoperable to receive recorded audio of the user administering medicationfollowing the displayed audio-video instructions.
 9. The medicationconfirmation apparatus of claim 8, wherein the processor is operable toanalyze the recorded audio to aid in determining proper medicationadministration.
 10. The medication confirmation apparatus of claim 1,wherein the audio-video instructions are part of a training sequence.11. A medication administration confirmation method, comprisingproviding a user feedback on how to properly administer medication inaccordance with a sequence of interactive instruction steps indicativeof proper medication administration employing a medicationadministration apparatus, as the user attempts to administer medicationbefore an image capture device, wherein providing the user feedbackcomprises: displaying one or more audio-video instructions on a displayof a local computing device for instructing a user to properlyadminister the medication in accordance with the sequence of interactiveinstruction steps indicative of proper medication administration, atleast one of the audio-video instructions instructing the user to followone or more steps of the sequence of interactive instruction steps toproperly position the medication administration apparatus relative to auser of the medication administration apparatus; capturing one or moreaudio-video sequences by a camera associated with the local computingdevice of the user administering medication following the display of theone or more audio-video instructions encouraging adherence to thesequence of interactive instruction steps including proper positioningof the medication administration apparatus for proper medicationadministration; in response to capturing the one or more audio-videosequences, automatically analyzing in real time, by a computer processorassociated with the local device, motion of the user and/or themedication administration apparatus within at least one of the capturedaudio-video sequences to confirm that the user has properly administeredthe medication in accordance with the sequence of interactiveinstruction steps employing the medication administration apparatus;determining, from the analysis, that the user has not followed one ormore steps of the sequence of interactive instruction steps, and thatmedication has not been properly administered; and responsive todetermining that the medication has not been properly administered,outputting to the display while the user is administering the medicationan additional one or more of the audio-video instructions to the user toencourage following of the one or more steps of the sequence ofinteractive instruction steps to confirm proper administration of themedication.
 12. The method of claim 11, further comprising outputting tothe display a prompt to the user to perform a particular sequence ofactions in real time in response to the displayed audio videoinstructions to be stored as the one or more audio-video sequences. 13.The method of claim 12, wherein the particular sequence of actions isdesigned to assist in determining proper medication administration. 14.The method of claim 13, wherein the particular sequence of actions inresponse to the displayed audio-video instructions further comprises:displaying a medication administration apparatus; positioning themedication administration apparatus; and administering the medication.15. The method of claim 11, further comprising transmitting from thelocal device one or more of the at least one audio-video sequences to aremote location for analysis.
 16. The method of claim 15, furthercomprising receiving further information indicative of whether the userhas properly administered the medication following the displayedaudio-video instructions from the remote location.
 17. A medicationtraining method, comprising providing a user feedback on how to properlyadminister medication in accordance with a sequence of interactiveinstruction steps indicative of proper medication administrationemploying a medication administration apparatus, as the user attempts toadminister medication before an image capture device, wherein providingthe user feedback comprises: displaying on a display of a localcomputing device one or more training audio-video instructionscomprising a sequence of medication administration instructions fortraining a user to properly administer a medication in accordance withthe sequence of interactive instruction steps indicative of propermedication administration, at least one of the training audio-videoinstructions instructing the user to follow one or more steps of thesequence of interactive instruction steps to properly position themedication administration apparatus relative to a user of the medicationadministration apparatus; capturing by the image capture device of thelocal computing device one or more audio-video sequences of each step ofthe user administering training medication via a medicationadministration training apparatus following the display of the one ormore training audio-video instructions encouraging adherence to thesequence of interactive instruction steps including proper positioningof the medication administration apparatus for proper medicationadministration; in response to capturing the one or more audio-videosequences, automatically analyzing in real time by a computer processorassociated with the local device motion of the user and/or themedication administration apparatus within of at least one of thecaptured audio-video sequences of at least one medication administrationtraining step to confirm that the user has properly performed the atleast one medication administration training step in accordance with thesequence of interactive instruction steps employing the medicationadministration apparatus; determining, from the analysis, that the userhas not followed one or more steps of the sequence of interactiveinstruction steps, and that the medication has not been properlyadministered during the at medication administration step; andresponsive to determining that the medication has not been properlyadministered, outputting to the display while the user is administeringthe training medication an additional one or more of the trainingaudio-video instructions to the user to encourage following of the oneor more steps of the sequence of interactive instruction steps toconfirm proper administration of the medication.
 18. The method of claim17, wherein the training medication to be administered via themedication administration training apparatus is an inhalable medication.19. The method of claim 18, wherein analyzing at least one of theaudio-video sequences comprises determining whether the inhalablemedication training apparatus is properly positioned and oriented. 20.The method of claim 17, wherein the medication to be administered viathe medication administration training apparatus is an injectablemedication.
 21. The method of claim 20, wherein analyzing at least oneof the audio-video sequences comprises determining whether theinjectable medication training apparatus is properly positioned andoriented.
 22. The method of claim 17, further comprising outputting tothe display a prompt to the user as to where and how to position themedication administration training apparatus in accordance with the oneor more training audio-video instructions during performance of the oneor more audio-video sequences.
 23. The method of claim 17, furthercomprising transmitting at least one of the captured audio-videosequences to a remote location for further analysis.
 24. The method ofclaim 17, further comprising notifying one or more of an administratorand the user responsive to determining that the user has not properlyadministered the training medication.
 25. The medication confirmationapparatus of claim 1, wherein the one or more audio-video instructionscomprise interactive instructions that allow the user to selectivelyview additional information regarding administering the medication. 26.The method of claim 11, wherein the one or more audio-video instructionscomprise interactive instructions that allow the user to selectivelyview additional information regarding administering the medication. 27.The method of claim 17, wherein the one or more audio-video instructionscomprise interactive instructions that allow the user to selectivelyview additional information regarding administering the medication.